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|. Introduction 


Ongoing research and evaluation that produce actionable, widely disseminated evidence are central to the 
continuous improvement of U.S. Department of Labor (DOL) programs. In addition, the Workforce 
Innovation and Opportunity Act of 2014 (WIOA) requires DOL to conduct periodic, independent 
evaluations to inform the effective operation of WIOA programs and services. To support the 
development of DOL’s research portfolio, this scan examines existing evidence on key topics related to 
WIOA programs and services. ! 


The scan focuses on four main topic areas, selected through a series of discussions with DOL’s Chief 
Evaluation Office, Employment and Training Administration (ETA), and other DOL staff: case 
management, integrated service delivery, training programs, and youth services. Although the first 
three sections on service strategies focus on evidence from studies on adults or general populations, these 
services also relate to programs serving youth. However, because of the policy importance of identifying 
effective strategies for disconnected youth ages 16 to 24, we devote the final section to research on youth, 
focusing on the many strategies offered by the WIOA Youth program. 


These topic areas were selected because they are viewed as critical to the success of job seekers and 
employers, and because they include interventions that offer the potential to be rigorously evaluated. The 
brief describes strategies associated with each of the four topic areas, discusses existing evidence on the 
effectiveness of those strategies and evaluations currently under way, and summarizes the knowledge 
gaps in the existing evidence base. Our scan prioritizes studies of strategies used in WIOA and by partner 
programs, but it also incorporates studies on related programs to increase the pool of studies from which 
to draw evidence. 


ll. Approach to research evidence scan 


Several foundational sources were used to identify the studies included in this brief. First, we drew upon 
large-scale studies on the Workforce Investment Act (WIA), WIOA, and American Job Centers (AJCs). 
The WIA Gold Standard Evaluation (Fortson et al. 2017) includes an impact and implementation analysis, 
whereas the Institutional Analysis of AJCs (Holcomb et al. 2018) and the WIOA Implementation Study 
(Sattar et al. 2020) were not rigorously tested and do not include an analysis of impacts. Second, we 
searched two workforce development research clearinghouses: the Clearinghouse for Labor Evaluation 
and Research (CLEAR)—specifically, for the reviews of apprenticeship and work-based training, 
opportunities for youth, and entrepreneurship and self-employment topic areas; and the Pathways to Work 
Evidence Clearinghouse, focusing on studies on coaching and career navigation. 


In addition, we also incorporated reviews that Mathematica previously conducted for DOL on (1) Job 
Corps (Berk et al. 2018), (2) the Reentry Employment Opportunities Evaluation (Lacoe and Betesh 2019), 
and (3) the Trade Adjustment Assistance (TAA) Behavioral Intervention and Long-Term Design Project 
(Goger et al. forthcoming). Finally, to expand on the literature covered in these reviews, we conducted a 
targeted literature search using terms based on the selected topics of interest in Google Scholar and 
Google from the websites of policy research firms with experience in workforce policy and from the ETA 
publications database. 


' This research evidence scan is one of a pair of reports developed as part of DOL’s WIOA Research Portfolio 
project. The companion report is “A Scan of Key Trends in the Labor Market and Workforce Development System” 
(Mack and Dunham 2021). 
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To identify a broad set of potentially promising strategies, we also incorporated studies on programs 
administered outside of WIOA, such as the Temporary Assistance for Needy Families (TANF) program 
administered by the HHS Administration for Children and Families or the Disability Program Navigator 
initiative administered by the Social Security Administration. Furthermore, this scan is not limited to 
studies that meet any particular evidence standard or other specific criteria, as would occur in a systematic 
evidence review for CLEAR or other clearinghouses. However, we do place greater emphasis on studies 
that used randomized controlled trial (RCT) and other rigorous designs. The scan also includes studies 
that found no statistically significant impacts of the strategy in question and discusses strategies that have 
used less rigorous evaluation designs. Finally, in synthesizing the available evidence, we highlight 
knowledge gaps where more evidence is needed. The knowledge gaps primarily focus on questions that 
could be answered by impact studies, though we also include questions that might be addressed by 
implementation research that could support such impact studies. 


In sum, the aim of this scan is to generate ideas on promising workforce development strategies that could 
be rigorously evaluated in the WIOA context. The scan casts a wide net to identify such strategies. It is 
not a formal evidence review using an established protocol. 


lll. Case management 


Case management, broadly defined, is one-on-one assistance or individual counseling focused on 
providing supports and information to assist individuals with immediate and long-term goal achievement. 
Under WIOA, this type of individualized assistance is one of several activities under the umbrella of 
Individualized Career Services (see Box 1). Case management often occurs before employment, and 
sessions include activities such as career planning, skills and needs assessments, job search assistance, 
and referrals for supportive services. The way in which these services are delivered and the involvement 
of program staff—sometimes referred to as employment counselors, coaches, career advisors or case 
workers—can impact participant success. Case management services are also a critical component of 
programs funded by WIOA Youth, but the services offered by these programs generally differ from those 
provided to adults and are discussed in the youth services section. 


The most direct and rigorous evidence on the 
effectiveness of WIOA-supported case 
management comes from the WIA Gold 
Standard Evaluation, a nationally ° 
representative RCT study on the Adult and 


Box 1. Individualized career services under the 
WIOA Adult and Dislocated Worker programs 


Comprehensive and specialized assessments of the skill 
levels and service needs 


Dislocated Worker programs (Fortson et al. 
2017). It found that job seekers assigned to 
receive intensive services—assistance 
through one-on-one counseling and 
workshops—achieved higher employment 
and earnings than those with access only to 
self-service resources. The intensive services 
group had earnings that were approximately 
$3,300 higher across 36 months than the self- 
services group—more than the costs of 
providing these services—and those 
receiving intensive services were more likely 
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Development of an individual employment plan 
Group or individual counseling 

Career planning 

Short-term pre-vocational services 

Internships and work experiences 

Workforce preparation activities 

Financial literacy services 

Out-of-area job search assistance and relocation 
assistance 

English-language acquisition and integrated education 
and training programs 


Source: 20 CFR 678.430 
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to be in jobs that offered fringe benefits. In another RCT study of models for delivering individual 
training accounts, participants randomly assigned to receive more intensive counseling on their choice of 
training had higher earnings in the follow-up period, which ranged from six to eight years following 
random assignment. This group also received training accounts that were approximately $1,800 dollars 
higher, on average; in addition, the more directive counseling they were supposed to receive was 
generally not delivered as planned (Perez-Johnson et al. 2011). 


A. Models of case management 


The ways in which case management is delivered can vary and be tailored to meet the needs of 
participants. Examples of different models of case management delivery and approaches can be identified 
across sectors, with promising strategies showing potential for job seekers. These include coaching 
models (drawn primarily from TANF programs with a focus on employment goals), intrusive advising (or 
proactive outreach to students to provide support and additional services) and navigator models 
(sometimes used with TAA participants), and strengths-based models (for supporting individuals with 
significant barriers). Overall, there has not been substantial research on how case management is 
administered and practiced in WIOA programs. 


For heuristic purposes, we describe case management models as distinct approaches. However, these are 
not mutually exclusive; for instance, coaching and navigator models may include strengths-based 
strategies for working with program participants. We provide examples to illustrate the differences in 
service delivery and the environments in which these approaches have been applied. Some models have 
roots in areas beyond workforce development, such as social services and rehabilitation settings, but 
promising aspects of these models could be applied to counseling and case management approaches 
within WIOA and other workforce programs. 


1. Coaching models 


In coaching models, staff work collaboratively with participants in a nondirective way to identify their 
goals and support their progress toward goal achievement. Unlike many case management approaches 
where staff set goals for participants and define what actions need to be taken to meet those goals, 
coaches act as a partner to support and motivate participants (see Box 2 for more detail on these models; 
Joyce and McConnell 2019). Coaching models have been used in a few WIOA programs and increasingly 
to engage with and support TANF program participants to improve employment and earnings outcomes, 
which makes these models especially applicable for expanded use with WIOA programs and services. 


Only a few studies have investigated coaching models, and the impacts have been mixed. The Mobility 
Mentoring model aims to help individuals attain and preserve economic independence by using goal 
setting, action plans, earned incentives, recognition, and self-belief and awareness (Babcock 2018). The 
Washington State Department of Early Learning piloted this model for parents of children attending Head 
Start. In an evaluation of this pilot, parents receiving Mobility Mentoring had higher earnings than those 
in a comparison group that did not receive Mobility Mentoring services (Babcock 2018). The Michigan 
Goal Progress approach, an approach focused on TANF applicants and recipients, focuses on identifying 
employment-related goals and breaking them down into actionable steps. However, an RCT found that 
the program did not have statistically significant impacts on employment, earnings, or public benefit 
receipt (Martinson et al. 2020). Additional coaching models focusing on TANF recipients, including the 
human services coaching model that has also been used in some WIOA programs, are discussed in Box 2. 
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2. Intrusive advising 


Often applied in an academic setting, intrusive advising involves advisors proactively engaging with 
students and encouraging them to be participants in the advising process, as opposed to being reactive and 
responding only to students’ requests for assistance. This model sometimes requires students to maintain 
attendance in advising sessions as a condition of enrollment in their academic program (Rajecki and 
Lauer 2007; Donaldson et al. 2016). In a workforce setting, a similar approach could include proactive 
outreach by case managers to provide career services to job seekers. In instances where a WIOA 
participant is also enrolled in a training program or where the AJC is co-located with a community 
college, for example, intrusive advising could be combined with services offered under WIOA. 


Box 2. Additional coaching models for TANF participants 

Derr and Joyce (2020) describe a set of coaching models from the TANF literature that are currently being 
evaluated: 

Goal4 It!™ This model focuses on moving from compliance-driven interactions to engaging, meaningful 
exchanges with participants to support and motivate them toward change. The model uses four steps—Goal, Plan, 
Do, Review/Revise—aimed at improving outcomes for children and families. It uses three main strategies: (1) 
reducing sources of stress, (2) strengthening life skills, and (3) forming relationships within and beyond the 
program. This model is currently included in the Evaluation of Employment Coaching for TANF and Related 
Populations. 

MyGoals for Employment Success (MyGoals). The MyGoals model is built on a sustained, three-year 
relationship between staff and the participant, focusing on helping participants achieve their goals by addressing 
challenges in executive functioning skills—the mental skills needed to complete tasks such as emotional control, 
stress tolerance, and time management (Riccio and Castells 2020). This model is currently included in the 
Evaluation of Employment Coaching for TANF and Related Populations. 


Family-centered coaching. The objective of this model is to support families holistically as they move toward goal 
achievement and economic prosperity. This includes a family-led and strengths-based approach with seven areas 
of focus: (1) child well-being and parenting; (2) employment, education, and career; (3) legal assistance; (4) 
financial assistance; (5) health and well-being; (6) family, friends, and relationships; and (7) basic needs—food, 
housing, transportation, and safety (Derr and Joyce 2020). 

Human services coaching. This model focuses on helping participants identify goals and drive motivation. 
Specifically, staff assist participants to direct, own, and experience the changes they want to see in their lives. Its 
core principles are to be person-centered, relationship-based, and goal-driven. 

Two additional coaching models are also included in the ongoing Evaluation of Employment Coaching for TANF 
and Related Populations: 

Family Development and Self-Sufficiency (FaDSS). Administered by the lowa Department of Human Rights, 
FaDSS focuses on assisting families at risk of long-term welfare receipt. This approach includes goal setting and 
ongoing support from a coach, all through a home visiting approach. Coaches conduct at least two home visits 
during the first three months of participation, followed by visits as needed with a minimum of one visit per month 
(Moore et al. 2019). 

LIFT. Based in New York City, Chicago, and Los Angeles, LIFT focuses on long-term financial goals of 
participants, such as a home purchase or college savings for a child. Coaches work with participants to set short- 
and long-term goals and identify milestones toward achieving those goals. As they move forward with their goals 
and complete a “three-month goal cycle,” participants can receive financial payments in the amount of $150, 
totaling up to $1,000 over a two-year period (Moore et al. 2019). 


Several RCTs have found that intrusive advising models improve degree completion among women 
(Evans et al. 2017), academic success (Abelman and Molina 2001; Kirk-Kuwaye and Nishida 2001), and 
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retention (Abelman and Molina 2001). In Jeschke et al. (2001), students who received intrusive advising 
as part of an RCT expressed a higher level of satisfaction with advising and more connectedness with the 
department, compared to participants in the control group that received traditional advising (such as 
regular but infrequent meetings on course selection). From a qualitative study of student opinions of 
intrusive advising, Donaldson et al. (2016) found that benefits of intrusive advising in an academic setting 
included greater participation in degree planning, increased relationship building with their advisor, and 
increased feelings of individualized support on the part of students. 


Intrusive advising has also been used as part of some Trade Adjustment Assistance Community College 
and Career Training (TAACCCT) programs. Cosgrove et al. (2019) conducted a regression analysis of 
student data from Missouri’s Manufacturing Workforce Innovation Network, a stackable credential 
program that included intrusive advising and supports for adult students. In this study, students who 
received intrusive advising were more likely to complete the program and receive a certificate. A quasi- 
experimental design (QED) study of another TAACCCT program, the Michigan Coalition for Advanced 
Manufacturing, also found that intrusive advising was associated with an increase in the likelihood of 
obtaining a nondegree credential for this same population of adult learners (Negoita 2019). 


3. Navigator models 


A career navigator provides guidance in assessing career interests, understanding various steps to pursue a 
selected career, developing training plans, and guiding participants toward appropriate services offered by 
partner and other programs (Center for Law and Social Policy 2016).* Although WIOA Youth programs 
include career navigators as a key service, this section focuses on evidence about programs for adults or 
general populations. Two examples of navigator models for adult populations include those implemented 
in academic settings as part of the TAACCCT programs and the Disability Program Navigator (DPN) 
initiative. In a number of quasi-experimental studies of the TAACCCT programs, students in an academic 
program receiving career navigation and supportive services were compared to a group of students who 
were in similar programs but were not offered these services. These studies found mixed evidence on the 
effects of career navigation and supportive services on the likelihood of employment and earnings (Jensen 
et al. 2016; Price et al. 2018; Dauphinee and Bishwakarma 2018; Maxwell et al. 2017; Rayyes et al. 
2016). In the DPN initiative, navigator staff (themselves known as DPNs) assist individuals with 
disabilities seeking employment services within local AJCs (then referred to as One-Stop Career Centers), 
with the ultimate goal of improving employment outcomes for people with disabilities (Livermore and 
Colman 2010). 


A descriptive study of the DPN model found that AJCs serve a large proportion of individuals receiving 
Social Security Administration (SSA) disability benefits and that SSA recipients who used DPN services 
had positive employment outcomes (Livermore and Colman 2010). Some state workforce agencies use 
TAA Navigators to guide participants through the benefits and training provided by TAA, but this 
strategy has not yet been evaluated within that program. 


4. Strengths-based models 


Strengths-based models, used in many settings including when working with individuals with significant 
challenges to employment, focus on the overall strengths of individual program participants and 
encourage self-efficacy in receipt of services (Brun and Rapp 2001). Prior studies of the effectiveness of 


? Where career navigators are discussed in relation to WIOA services, this title appears to be synonymous with 
caseworker or counselor. 
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this model for individuals with serious mental health challenges have found positive effects in multiple 
areas, including employment, independent living, and reduced hospitalizations (Rapp and Sullivan 2014). 


B. Post-employment counseling 


WIOA programs, including programs supported by WIOA Youth, offer follow-up services that may 
include post-employment counseling. In instances where participants obtain employment, post- 
employment counseling may help support their transition to and ongoing success in their new position. 
These approaches may incorporate strategies from one or more of the case management models discussed 
previously. Although models that include post-employment counseling have been rigorously evaluated 
(several of them are discussed below), there is no clear evidence on the independent impact of post- 
employment counseling, separate from the effects of training or subsidized employment. The following 
are three examples of workforce development programs that include a post-employment counseling 
component: 


The Work Plus program, focused on TANF recipients as part of the Employment Retention and 
Advancement project, offered post-employment job supports and intensive case management to help 
participants find and retain employment. Although this program supported TANF recipients, the 
strategies it used are inherently workforce development strategies that could be applied to WIOA 
programs. The authors conducted an RCT in which participants were assigned to the treatment group (to 
receive employment services through Work Plus) or the control group (to receive standard TANF 
services). Evidence suggests that there were no statistically significant effects on employment or earnings 
for Work Plus participants as compared to the control group (Navarro et al. 2007; U.S. Department of 
Labor 2016). 


The WorkAdvance program provided training and employment services to low-income adults to improve 
employment outcomes and meet the needs of local employers. The program included five main elements: 
screening of potential participants before enrollment, work-readiness services (for example, career 
coaching, supportive services), occupational training, job development and placement, and follow-up 
retention service coordinated with employers (U.S. Department of Health and Human Services 2020a; 
Hendra et al. 2016; Schaberg 2020; Tessler 2013; Tessler et al. 2014). Although the program overall 
demonstrated positive short- and long-term impacts, the evaluations were not designed to assess the 
effectiveness of the post-employment counseling component independently. 


The Good Transitions program, focused on low-income, noncustodial parents, placed participants into 
subsidized employment while providing job coaching, job development, and case management. After one 
month, participants were placed into a different position, with less coaching but continued case 
management and job development support. In an RCT of this program, Good Transitions program 
participants had increased short- and long-term earnings, increased employment, increased education and 
training completion, and decreased public benefit receipt compared to individuals in a control group who 
did not receive Good Transitions program services but could participate in other services from the 
Division of Child Support Services’ Fatherhood Program (U.S. Department of Health and Human 
Services 2020b; Barden et al. 2018; Bloom 2015; Fink 2018; Redcross et al. 2016). Again, however, 
further research is needed to understand the independent effects of counseling and case management, as 
distinct from the effects of subsidized employment. 


The Individual Placement and Support (IPS) model, in addition to job search assistance and a range of 
other services, also includes post-employment support. IPS is an approach to supported employment, 
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often applied in the vocational rehabilitation realm for individuals with significant challenges to 
employment, such as serious mental illness. IPS focuses on integrating vocational and treatment services, 
with the core principles focusing on competitive employment, systematic job development, rapid job 
search, integrated services, benefits planning, zero exclusion, time-limited supports, and worker 
preferences (IPS Employment Center 2021). Receipt of IPS has generally been shown to result in higher 
rates of competitive employment, more days worked, and greater earnings over the same period of time as 
compared to nonparticipants or those who received a limited range of services (Rosenheck and Mares 
2007; Bond et al. 2015; LePage et al. 2016; Ottomanelli 2012; Davis et al. 2012; Twamley et al. 2008). 


Much of the research on IPS programs focuses on IPS services provided to people with serious mental 
illnesses, but the model has been used in different settings, such as in TANF and workforce settings. 
However, the findings from these studies have been mixed. In a study of an IPS model provided to TANF 
recipients with physical and mental disabilities, those who received the IPS services had higher rates of 
competitive employment and earned more on average than the control group during the first year of 
follow-up (Farrell et al. 2013). In another study of IPS used in California’s TANF program, employment 
rates increased three times as much as the rates of those who received other mental health services offered 
by the agency (Chandler 2017). However, an RCT study of an IPS program delivered by San Diego 
County’s Workforce Investment Board found that those in the control group achieved higher rates of 
employment than those who received the IPS program services (Freedman et al. 2019). 


C. Knowledge gaps related to case management 


Some evidence, such as the WIA Gold Standard Evaluation, suggests that the receipt of case management 
can improve employment and earnings in a workforce development setting, though a number of related 
strategies have less of an evidence base. However, little is known about which services or elements of 
case management are responsible for positive impacts. In addition, this review did not find any evidence 
on the efficacy of various assessments at identifying interests, strengths, and challenges. Specific gaps in 
the current literature include the following: 


What case management models are currently in use in WIOA programs and services? What additional 
case management models could be applied to serving job seekers in WIOA programs? 


What are the most effective case management models, and in what settings? What activities should be 
included during case management sessions? What strategies are most successful for providing case 
management (for example, dosage and format of case management sessions)? 


How does the administration of case management models vary by state and locality, such as the 
availability of one-on-one services, time spent with counselors, level of intensity of counseling sessions, 
and specific types of services? How does that variation impact participant outcomes? 


Could navigator, coaching, intrusive advising, strengths-based, or trauma-informed models developed in 
contexts outside of WIOA be effective for WIOA participants? Which models would be most effective? 
In which situations? 


What is the independent effect of case management or one-on-one counseling, either pre- or post- 
employment, both generally and in the context of WIOA programs? In terms of post-employment, does 
the level of coordination between counseling staff and employers impact the effectiveness of post- 
employment counseling? 
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Are there particular models for the training of case managers that lead to more effective case 
management? Are there particular qualifications or characteristics that lead to more effective case 
managers? 


How effective are assessments in assessing skills and aptitudes and identifying employment challenges 
and career interests? What are the most effective assessments? 


IV. Integrated service delivery 


Creating more closely integrated services across programs has been an important principle of WIOA, as 
well as the goal of many recent initiatives aimed at generating efficiencies and reducing challenges in 
reaching self-sufficiency for individuals and families (U.S. Department of Labor 2021). Under WIOA, the 
One Workforce approach aims to create “greater collaboration, integrated service delivery, shared data, 
and leveraged resources that leads to positive employment and training outcomes for customers” (U.S. 
Department of Labor 2021). Integration of services and programs can impact all areas of operations, 
including case management approaches, training, staffing, funding, and administration. Additionally, 
integration aims to reduce siloing, or separation and isolation, of programs and services, thus supporting 
increased access and improved efficiencies. Conceptually, integrated service delivery applies to all WIOA 
programs and populations, including WIOA Youth programs. Although some of the evidence in this 
section discusses programs for adults or parents, two key studies—the Institutional Analysis of AJCs and 
the WIOA Implementation Study—focused on system operations, where those systems implement 
programs for both adults and youth. 


Integration could also help participants get benefits and services they are eligible for. For example, a 
family in need of assistance through the Temporary Assistance for Needy Families (TANF) program 
might receive contact information for the local TANF agency; however, the family may be responsible for 
initiating the process, including contacting the agency and applying for services. In an integrated model, 
WIOA staff may be co-located with TANF staff, the referral process could happen in real time (for 
example, following a career-focused counseling session, the family can immediately meet with TANF 
staff on-site), and coordination between the agencies may include cross-program integrated application 
and eligibility determination. This integrated approach, implemented through AJCs, is intended to help 
participants get served in a more holistic way, increasing the likelihood that their specialized needs are 
being met, breaking down barriers to obtaining additional services, and increasing their chances of 
successfully pursuing their educational and career goals. However, there have been few rigorous impact 
studies that aim to identify the effects of integrating services.? 


A. Strategies that aim to enhance co-enrollment and coordination among WIOA 
programs 


As of 2018, many AJCs had implemented strategies that do not require larger structural or policy 
modifications, such as co-location (at least part-time) among multiple partners, AJC-wide staff meetings, 
staff cross-training, coordinated outreach activities, and a streamlined intake process for program 


3 There are some studies of integrated services from human services agencies. In a study focused on integrating 
TANF services with mental health supports in Minnesota, in most sites, participants served after the integration 
initiative was implemented did not have better employment or earnings outcomes than those served before, after 
adjusting for other characteristics (Martinson et al. 2009). The Assessing Models of Coordinated Services for Low- 
Income Children and Families study is currently exploring further integrated services strategies (U.S. Department of 
Health and Human Services 2020c). 
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participants (Holcomb et al. 2018). Although these strategies have not been subject to rigorous evaluation, 
we describe them and their evidence base below. 


Co-location, consolidating administrative structures, and cross-training. The U.S. Government 
Accountability Office (GAO 2011) identified both co-location (that is, being located within the same 
building or facility) and consolidating administrative structures as ways to increase efficiency and 
improve coordination. At the agency level, 14 states consolidated core WIOA programs under a single 
agency, thus reducing or removing barriers to coordination. Additionally, in an implementation study by 
English and Holcomb (2020), the 14 states said the collaboration that occurred between core agencies 
during development of the state plan facilitated ongoing communication between these agencies and 
individual staff members. Finally, states noted opportunities for ongoing cross-training of staff across 
Title I programs (English and Holcomb 2020). GAO (2011) also notes that in three states where the 
welfare and workforce agencies have been consolidated—Florida, Texas, and Utah—state officials 
described realizing reduced costs and improved services. 


Box 3. WIOA partner programs 


WIOA core programs 


Title |: WIOA Adult, Dislocated Worker, and Youth 
Title Il: Adult Basic Education and Literacy 

Title Ill: Wagner-Peyser Employment Service 
Title IV: Vocational Rehabilitation 


WIOA required partners 


Department of Labor: Job Corps, YouthBuild, Indian 


and Native American programs, National Farmworker 
Jobs Program, Senior Community Services 
Employment Program, TAA, Unemployment 
Insurance, Jobs for Veterans State Grants, and 
Reentry Employment Opportunities 


Department of Education: Carl D. Perkins Career and 
Technical Education programs 


Department of Housing and Urban Development: 
Employment and training programs 


Department of Health and Human Services: 
Community Services Block Grant employment and 
training programs and TANF (newly added required 
partner under WIOA) 


Other optional partners (not exhaustive) 


Social Security Administration: Ticket to Work 
Department of Agriculture: SNAP Employment and 
Training and Able-Bodied Adults Without Dependents 
Corporation for National and Community Service: 
AmeriCorps 

Other appropriate government or private sector 
programs, such as libraries and child care providers 


Source: WIOA Sec. 121(b) 
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At the local level, the emphasis on co-location, as 
well as alignment and program coordination 
under WIOA, enabled partners to work together 
and streamline services (Dunham et al. 2020). 
Local AJC staff note that co-location was needed 
in order to implement a functional staffing model 
(where staff complete tasks based on function as 
opposed to employer) or hybrid model (a 
combination of the functional model and the 
specialized staffing model, where staff have 
specific responsibilities that do not overlap; 
Koller and Paprocki 2015). Additionally, co- 
location allowed for greater coordination between 
staff and a stronger referral process (Koller and 
Paprocki 2015). Co-location across AJCs in the 
WIA Gold Standard Evaluation varied, with 
some local areas having a different mix of 
partners co-located at the AJC. For example, 
Wagner-Peyser Employment Service (ES) was 
co-located in 27 AJCs, Senior Community 
Services Employment Program (SCSEP) in 16 
AJCs, and Job Corps in 12 AJCs; additionally, 
some states require certain partners to be co- 
located within each AJC, such as WIA, ES, 
Veteran's Employment and Training Service 
(VETS), Adult Education, and Vocational 
Rehabilitation (Koller and Paprocki 2015). In an 
earlier study, GAO (2011) found the benefits of 
co-location and consolidating administrative 
structures included reduced administrative costs 
and, therefore, the freeing of these funds for other 
uses, such as training additional customers. 
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The way in which AJC staff are organized also play a key role in coordination between programs. One- 
quarter of AJCs had WIA or ES staff conduct initial customer intake and enrollment, resulting in 
functional alignment of these activities across programs or operating agencies (D’ Amico 2015). Holcomb 
et al. (2018) note that most AJCs included in the Institutional Analysis of American Job Centers 
continued to be organized by program, rather than by function. ETA encourages states to cross-train staff 
in ES, WIOA, and the Trade Adjustment Assistance (TAA) programs (U.S. Department of Labor 2020). 


Durham et al. (2019) describe three models of coordination between WIOA, child care, and other related 
programs. In the least integrated model, the parallel model, states operate WIOA, Child Care and 
Development Funds (CCDF), and other related programs from different agencies. Parents in WIOA 
training programs may receive priority of service for child care access. In the collaborative model, states 
continue to operate programs in separate agencies but aim to maximize coordination between these 
agencies through active communication. At the greatest level of coordination, the integrated model, states 
bring all related workforce development and supportive services programs within a single agency. For 
example, the Texas Workforce Commission operates all six WIOA core programs, CCDF, TANF, and 
SNAP programs, and a single division develops and disseminates workforce development- and child care- 
related policies. 


Shared data and data systems. ETA encourages states to consider implementing integrated case 
management data systems across WIOA partner programs (U.S. Department of Labor 2020), and some 
research supports data-sharing policies to increase participants’ access to different benefits (Adams and 
Spaulding 2018; Durham et al. 2019). Koller and Paprocki (2015) note the benefits of shared data systems 
between partners within AJCs include reducing burden related to data entry and increased ability to track 
services provided to customers across other programs. Many AJC operators and managers do not have 
access to data from partner programs; partners operating different programs frequently do so with 
separate performance measures and data systems (Dunham et al. 2020). Similarly, program staff often 
lack cross-system access and, as a result, must complete data entry into multiple systems. In these 
instances, staff frequently used work-around methods to address these limitations, such as documenting 
customer information in spreadsheets (Holcomb et al. 2018). 


Pooled funding and cost sharing. Under WIOA, AJC partners must contribute to infrastructure costs, 
whether or not they are co-located. As of 2018, this was not the practice in many AJCs, and the 
expectation for contributing toward infrastructure costs corresponded directly with co-location (Brown 
and Holcomb 2018). Further, many partnerships used informal resource-sharing agreements (Holcomb et 
al. 2018). Holcomb et al. (2018) note, however, that although there may be resistance to formalizing these 
arrangements, these approaches may also solidify commitment among partners and encourage a greater 
presence within the AJC. Most recently, English and Holcomb (2020) found that, of the AJCs included in 
the WIOA Implementation Study, the memorandum of understanding documents and infrastructure 
funding agreements required under WIOA have increased the number of partners making financial 
contributions to the AJCs. However, co-location between partners did not change substantially, partners 
faced resource constraints that limited their ability to enter into such agreements, and establishing these 
agreements resulted in increased administrative burden (English and Osborn 2018). 


B. Effectiveness of supportive services provided by partner programs 


One of the primary ways integrated services could potentially improve participant outcomes is by 
connecting job seekers to the benefits and supportive services they are eligible for. Under WIOA, 
supportive services may include linkages to community services; assistance with transportation, child 
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care, or housing; needs-related payments, assistance with educational testing, accommodations for 
individuals with disabilities; legal aid services; referrals for health care; and assistance with work-related 
supplies (such as uniforms), educational items (such as textbooks and school supplies), and payments for 
employment- and training-related applications, tests, and certifications (20 CFR § 680.900). Many of 
these services are available on-site (at the AJC) or through referral to a partner program located outside of 
the AJC. However, McCutcheon and Mastri (2015) found that AJCs use only about 3 percent of their 
funds on supportive services. WIOA Youth also funds supportive services, though programs delivering 
these services specifically to youth are discussed in the youth services section. 


There have not been many rigorous evaluations of supportive services, which may reflect some 
challenges in evaluating them rigorously. “Supportive services” does not imply any single bundle of 
services because it is a set of supports that are available as needed. For example, for a participant who has 
children, the intervention may be child care, but for someone without a vehicle, it will likely be assistance 
with transportation, such as a bus pass. In a quasi-experimental study, among older workers, those in 
Workforce Investment Areas with a higher rate of providing supportive services (except needs-related 
payments) also entered the workforce at a higher rate (Zhang 2011). Several reviews of the existing 
literature suggest that participants receiving supportive services, in combination with technical training 
and case management, have more positive employment and earnings outcomes compared to groups that 
do not receive such services (Davis et al. 2013; Gueron and Hamilton 2002; Maguire et al. 2010; U.S. 
Department of Labor et al. 2014). However, little is known about the independent effect of these services. 


C. Knowledge gaps related to integrated service delivery 


There have been a number of efforts and strategies implemented to improve integration of workforce 
services—many documented in implementation and descriptive studies—but there is limited rigorous 
evidence on the effect of integrated services that are closely related to WIOA. There may be challenges 
with implementing a rigorous evaluation of integrated services models or programs. In particular, 
integrated services is inherently a system-level intervention, making it challenging to compare groups of 
participants who are in the same system and are thus facing similar labor market conditions. One of the 
purposes of integrating services is to allow participants to more easily access additional needed services, 
but there is limited evidence on the impact of these services. 


Specific gaps in the current literature include the following: 


e Is service integration an effective strategy for improving outcomes of program participants? Which 
aspects of integration are effective at meeting goals of an integrated approach, such as reducing 
barriers for participants or increasing access to partner programs? 


e What is the level and type of integration seen at the state and local levels among WIOA programs? 
Does state-level coordination impact local-level service delivery, and if so, how? 


e For integrated services delivered in other contexts, what aspects of service delivery could be applied 
to WIOA? 


e Which aspects of WIOA, including service delivery strategies, would benefit most from integrating 
services? Are certain aspects or strategies more challenging or costly to integrate? 


e What populations served under WIOA would benefit most from receiving services under an 
integrated approach? 
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e Does increased availability and access to supportive services (both additional types of services and 
expanded funding for these services) impact employment and earnings outcomes for job seekers? 


V. Training models 


The most rigorous study of the federal workforce development legislation and related programs was an 
RCT conducted as part of the WIA Gold Standard Evaluation, which focused on the predecessor to 
WIOA. Although not conclusive, the study found that training funded by the Adult and Dislocated 
Worker programs did not have positive impacts in the 30 months after study enrollment. The inconclusive 
results could be explained by various factors such as low take-up of WIA-funded training in the treatment 
group and the control group receiving non-WIA-funded training (Fortson et al. 2017). In an older RCT 
study on the Job Training Partnership Act, the predecessor of WIA, trainees earned $2,598 more, on 
average, than those who did not receive program services two years after program completion (Nudelman 
2000). 


A number of quasi-experimental studies of programs providing training to low-income adults, 
implemented under WIA, have demonstrated small but statistically significant impacts on earnings and 
employment. Such impacts have been documented in the short term (less than a year, McEntaffer 2015) 
and for 12 calendar quarters following WIA registration (Andersson et al. 2013; Heinrich et al. 2009; 
Hollenbeck 2009). Other studies have found that employment impacts decay over time (Greenberg et al. 
2003; Bloom et al. 1997). Studies examining impacts for subgroups have observed that earnings impacts 
are generally more positive for adult women than men (for example, D’Amico 2006) and might vary with 
the type of training received. For example, classroom training tailored to specific occupational skills and 
on-the-job and employer-provided training were found to be more effective than occupational skills 
training programs without those components (D’ Amico and Salzman 2004). 


In the remainder of this section, we discuss the evidence base on different types of training programs and 
models utilized by the public workforce system, such as work-based, classroom-based, and others. 
Training provided through WIOA Youth programs is discussed in the next section. That said, a number of 
strategies discussed here are relevant to training programs for youth, and young adults are provided 
training in most of the programs and models discussed in this section. 


A. Work-based learning 


Work-based learning program models, also referred to as “earn and learn” models, provide participants 
with opportunities to apply general skills related to holding a job in a real-world setting. Work-based 
learning programs may be paid or unpaid, and they may or may not include a commitment from an 
employer partner to hire participants after completing the training component or be offered to people 
while they are in training (Heinrich et al. 2009). Common examples include apprenticeship, on-the-job 
training, paid internships, “returnships,” and other earn-and-learn strategies.* 


1. Apprenticeships 


Apprenticeship models involve an industry- and employer-driven structured approach to occupational 
training. Apprenticeship models typically combine paid, work-based learning; on-the-job training and 
mentorship; related technical instruction, often provided in a classroom setting; and an industry- 
recognized credential upon completion. Apprentices are paid, productive employees of an employer that 


4 The majority of the evidence on internships is on those provided to youth and is discussed in a later section. 
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sponsors or partners with the apprenticeship program. Apprenticeship has long been dominated by the 
construction trades, and the existing evidence has largely been focused in these trades. However, DOL 
and state apprenticeship offices have recently focused on expanding apprenticeship programs in sectors 
with high demand for skilled workers, most notably health care, information technology, and advanced 
manufacturing. 


Although the evidence base for apprenticeship programs is limited, there is evidence of registered 
apprenticeship programs improving the earnings outcomes of program participants. For example, a QED 
study of a registered apprenticeship program found that employment rates were 8.6 percentage points 
higher and average annual earnings were $6,595 higher for those who participated in the registered 
apprenticeship than those who did not (Reed et al. 2012). In another QED study of a registered 
apprenticeship program in Washington State, those who participated in the apprenticeship program earned 
$3,500 more in quarterly earnings compared to non-apprentices (Hollenbeck and Huang 2016). The 
Apprenticeship Evidence-Building Portfolio, a large, national QED impact study of apprenticeship 
programs funded by DOL, is currently underway.° 


2. Transitional employment 


Transitional jobs, also referred to as subsidized jobs, are a form of earn-and-learn employment program 
that provide participants with work experience and help them find employment by paying all or some of 
their wages. Transitional employment programs are not always included with other work-based learning 
models, because these programs offer a mix of strategies. For example, some transitional employment 
programs provide only work-based income support, whereas others aim to improve future employability 
by addressing employment-related barriers such as educational attainment and vocational skills. 
Transitional employment programs may often provide classroom training, job search assistance, and basic 
education services. They may also conduct job readiness assessments and tailor services based on career 
interests or by allowing participants to explore different vocational training opportunities (Sattar 2010). 


Although a few studies of transitional and subsidized employment programs have demonstrated positive 
long-term earnings and employment impacts, the literature overwhelmingly suggests that the impacts of 
such programs are mostly observed during the period of the transitional job placement and fade afterward 
(Bloom 2014; Butler et al. 2012; Cook et al. 2015; Fontaine et al. 2015; Glosser et al. 2016; Jacobs 2012; 
Redcross et al. 2016; Roder and Elliot 2012; Rotz et al. 2015; Uggen 2000; Walter et al. 2017). 


One RCT study examined the Personal Roads to Individual Development and Employment (PRIDE), 
which provided employment services to participants while also taking into account their medical 
conditions when assigning them to activities. The study found large and significant impacts on earnings 
and employment during the first year after random assignment and an impact of 4 percentage points on 
employment during the ninth quarter after random assignment (Bloom et al. 2007). Another RCT 
examined the Enhanced Transitional Jobs Demonstration (ETJD)—here, the program increased 
participants’ earnings and employment rates 30 months after enrollment, when most participants had left 
their transitional jobs. The enhanced transitional jobs programs served people recently released from 
prison or unemployed non-custodial parents who had fallen behind in child support payments (Barden et 
al. 2018). Finally, Cummings and Bloom (2020) synthesized the findings from RCTs of 13 subsidized 
employment programs that were part of the Health and Human Services (HHS) Subsidized and 


5 Other apprenticeship studies under way include the following: Impact Evaluation of Teacher Residency Programs 
and the Rural Youth Apprenticeship Development Project, Expanding Apprenticeship Pathways for Youth and 
Adults with Disabilities, and the California Apprenticeship Initiative Technical Assistance and Evaluation. 
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Transitional Employment Demonstration (STED) and the ETJD. About half of the 13 programs 
demonstrated improved employment and earnings impacts through the second year after study 
enrollment, and four programs sustained earnings improvements beyond the second year (Cummings and 
Bloom 2020). 


Additionally, a report from the Center for Employment Opportunities (CEO) presented findings from a 
QED study of their transitional job programs serving individuals with a recent criminal history in New 
York. The report states that CEO participants in the study were approximately 50 percent more likely to 
be employed than their counterparts in the comparison group, and that these impacts persisted through 
three years following enrollment (Center for Employment Opportunities 2019). 


B. Career pathways and sector-oriented training 


Career pathways programs allow participants to progress through education and training in multiple 
discrete, stackable components in particular sectors by offering a clear sequence of education coursework 
or training credentials aligned with employer-validated work readiness standards and competencies 
(Kozumplik et al. 2011). Career pathways programs under WIOA are defined as programs that offer a 
clear sequence, or pathway, of education coursework and/or training credentials aligned with employer- 
validated work readiness standards and competencies, and are guided by six essential components. The 
six components are: (1) building cross-agency partnerships and clarifying roles, (2) identifying industry 
sectors and engaging employers, (3) designing education and training programs, (4) identifying funding 
needs and sources, (5) aligning policies and programs, and (6) measuring system change and performance 
(U.S. DOL and Manhattan Strategy Group 2015). 


Overall, the evidence on the effectiveness of these programs is mixed. A literature review that included 96 
impact evaluations of career pathways programs identified that 62 percent of the studies that measured 
employment outcomes found positive impacts, but 33 percent found no impact (Sarna and Adam 2020). 


For example, the Green Jobs and Health Care Impact Evaluation, an RCT of four career pathways 
programs, found positive earnings and employment impacts for a program in the clean energy sector that 
included stackable credentials and supportive services, but the other programs were not effective 
(Martinson et al. 2016). The ShaleNET evaluation found that the ShaleNET noncredit programs had a 
positive and significant impact on employment in the first, second, and third quarters after completion 
when compared to the nontraining comparison groups (Dunham et al. 2016). An RCT study of the 
Accelerated Training for Illinois Manufacturing program that incorporated career pathways observed 
positive impacts on earnings and employment in the second year after random assignment (Betesh et al. 
2017). Interim findings from an evaluation of the Pathways for Advancing Career and Education 
observed increases in the percentage of participants working in a mid-skill job (Farrell and Martinson 
2017). However, in an RCT of a DOL Workforce Innovation Fund grantee serving young adults and 
including work-based learning and career pathway approaches, participants had statistically significant 
lower earnings than the control group two years after random assignment (Geckeler et al. 2017). 


A recent synthesis report of the impact findings from 23 third-party evaluations using quasi-experimental 
study designs to estimate the impact of the TAACCCT projects, which were based on a career pathway 
model, detailed some positive impacts on labor outcomes. The synthesis identified that 6 of the 11 studies 
examined observed positive employment or earnings impacts (Kuehn and Eyster 2020). Box 4 highlights 
some of the findings from “A Synthesis of Impact Findings from the Trade Adjustment Assistance 
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Community College and Career Training (TAACCCT) Third-Party Evaluations” (Kuehn and Eyster 
2020). 


Box 4. Findings from “A Synthesis of Impact Findings from the Trade Adjustment 
Assistance Community College and Career Training (TAACCCT) Third-Party 
Evaluations” 

The TAACCCT grant program awarded $1.9 billion to institutions of higher education offering programs of two 

years or less to build their capacity to provide workforce education and training to adults in need of new skills for 

in-demand jobs. The grant program also sought to change education and workforce systems to more effectively 
address employer needs for skilled workers and transform how community colleges deliver education and training 
to adult learners. In addition to the improved training, the grantees were required to conduct local evaluations of 
their programs. The Urban Institute synthesized the impact findings from 23 local evaluations. The synthesis notes 
the following positive impacts: 

e The study of the Advanced Manufacturing, Mechatronics, and Quality Consortium project to enhance advanced 
manufacturing and career training found that employment rates increased by 38.9 and 54.2 percentage points 
in the third and fourth quarters after program enrollment (Negoita et al. 2017). 

e The evaluation of the Gateway Community and Technical College project IMPACT found that the project raised 
quarterly earnings increases from the pre-enrollment to the post-enrollment period by $3,133 (Jensen et al. 
2016). 

However, one of the evaluations observed negative earnings impacts. The evaluation examined the North Dakota 

Advanced Manufacturing Skills Training Initiative and found that program completers were less likely to have 

earnings increase after completion (WorkED 2017). 


1. Sector-oriented training 


Sector-oriented training programs are related to career pathways programs and aim to provide training 
that produce workers with the skills needed by groups of employers in a particular industry or sector that 
has strong local demand and offers the opportunity for career advancement (Schaberg and Greenberg 
2020). These programs are often provided through partnerships between employers and training and 
educational organizations. Input from employers and published labor market information are used to 
design these programs, and employers also provide work-based training opportunities, such as internships 
or on-the-job-training, to participants (Holzer 2015). Many sector-oriented training programs also provide 
participants with job search assistance and job placement services, and some programs continue working 
with participants after they find a job (Schaberg and Greenberg 2020). Common industries targeted by 
sector-oriented training programs include health care, information technology, manufacturing, and 
transportation (Schaberg and Greenberg 2020). 


Three studies of sector-oriented programs have observed positive long-term earnings impacts. Three of 
four programs using the WorkAdvance model—a model that that provides employee assessments, career 
readiness services, occupational skills training, and job development and placement services in different 
sectors—have demonstrated positive impacts on earnings two years after program completion (Hendra et 
al. 2016; Schaberg and Greenberg 2020). In another RCT study, three sector-based training programs had 
impacts of $4,500 over the 24-month study period (Maguire et al. 2010). Early findings from the Health 
Professions Opportunity Grants RCT study suggest that the treatment groups’ wages were 4 percent 
higher than those in the control group in the fifth and sixth quarters after random assignment (Martinson 
et al. 2016). The evaluation also suggests that the impacts were larger for those with some college or a 
degree and with fewer barriers to employment before the program began (Martinson et al. 2016). In 
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addition to positive impacts on participant earnings and wages, qualitative descriptive studies of sector- 
based programs have also been found to improve employer satisfaction with the trainees because of 
reduced turnover, absenteeism, recruitment costs, and customer complaints (Lea 2004). 


C. Incumbent worker training 


Incumbent worker training (IWT) programs provide those who are currently employed with services such 
as mentoring, on-the-job learning, and other training to help with career advancement and job retention. 
Under WIOA, IWT programs focus on ensuring that employees of a company can acquire the skills 
necessary to retain employment and advance within the company, or to acquire skills necessary for 
averting a layoff (WIOA 2017b). Few studies have rigorously examined such programs, but findings from 
two quasi-experimental studies demonstrate some positive impacts related to job retention and job 
advancement. One study that analyzed a formal training program found that training hours were 
positively related to frequency of promotion over a two-year period (West 2010). A second QED study of 
a California state-funded incumbent worker program observed positive impacts for employers, such as 
improved company sales and firm size (Negoita and Goger 2020). 


D. Strategies for improving training completion 


The WIA Gold study identified several challenges with WIA-supported training: 40 percent of those 
offered training completed their programs, and of those offered training and entering an occupation- 
specific training program, 41 percent found employment related to their training (Fortson et al. 2017). 
Although the research examining strategies for improving training completion is limited, three non- 
experimental studies have identified promising strategies for improving training completion in career 
pathways programs. A study of Wisconsin’s Making the Future Initiative in advanced manufacturing 
examined whether embedding short-term certificates or credentials within longer-term “parent” programs 
led to increased program completion rates. Students enrolled in career pathway programs with stacked 
and latticed credentials were far more likely to complete nondegree credentials of various lengths (for 
example, one-year technical diploma, less than one-year technical diploma, and local or career pathway 
certificate) and were also more likely to earn multiple credentials (Price et al. 2018). 


A study of the Health Professions Pathways found that exposing students to holistic supports led to a 
statistically significant positive impact on credential attainment. The holistic student supports offered to 
students in this study focused on seven categories: academic advising, assessments, career services, 
employment services, financial services, nonacademic advising, and social services and counseling (Giani 
et al. 2018). Additionally, in a study of the Wisconsin Technical College System, adults receiving student 
supports were much more likely to complete a nondegree credential compared to adult students in the 
matched comparison group. The student supports examined included academic supports (such as 
enhanced classroom instruction, tutoring, and test preparation), as well as nonacademic supports (such as 
personal counseling and case management, job search and placement, and study skills and time 
management). Some of these supports were integrated into program curriculum or embedded in the 
classroom, while others were delivered outside of class through both one-on-one and group-level sessions 
(Valentine and Price 2019). 


E. Knowledge gaps related to training models 


The WIA Gold Standard evaluation found that training provided through WIOA Adult and Dislocated 
Worker programs did not detect statistically significant impacts. There has been an increased focus 
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among workforce development programs on work-based learning strategies, but there have not been 
enough rigorous studies of these strategies, and relatively few WIOA participants currently receive work- 
based learning. There are some promising findings from recent evaluations of career pathways and sector- 
based training, whereas transitional jobs have generally been found to not have impacts in the long term. 
Some of the gaps that could be examined in future studies include the following: 


e What are the impacts of work-based learning strategies such as apprenticeship and on-the-job training 
implemented under WIOA? 


e What aspects of work-based learning models and career pathways and sector models can be adapted 
under WIOA? 


e What types of employer partnerships help make work-based learning successful? How much do 
employer partnerships contribute to impacts? 


e What initiatives help increase training completion, obtaining credentials, and obtaining jobs in the 
field of training? 


e What are the impacts of including additional supportive services along with employment training 
models? Do these services help different types of job seekers participate in and complete training? 


e What types of strategies could make the impacts of transitional and subsidized employment programs 
persist in years following program completion? 


VI. Youth services 


WIOA Youth programs incorporate the 14 program elements specified in the Act (Box 5).° Because the 
WIOA Youth funding stream supports many different programs and services, there is not a single WIOA 
Youth program that could be rigorously evaluated. However, there have been several evaluations of 
programs serving youth populations. Many of these programs include components similar to those funded 
through WIOA Youth, or have used WIOA Youth funds to support their services. For example, as 
detailed below, programs such as Job Corps and YouthBuild have demonstrated some positive impacts on 
labor outcomes for youth. In this section, we review the evidence on services typically supported by 
WIOA Youth, as well as other programs that serve youth populations. In addition, many programs 
supported by WIOA Youth funds deliver services that are the same or similar to those of non-WIOA- 
supported programs reviewed in this section. 


Job Corps, supported under the WIOA legislation, is the nation's largest and most comprehensive career 
technical training and education program for at-risk youth. The program's objective is to help 
disconnected youth become more responsible, employable, and productive citizens by providing them 
with individualized academic education, vocational training, counseling, and job placement assistance. A 
defining feature of Job Corps is its residential component: Most participants live in a Job Corps center 
during the week, where all education, training, and counseling services are provided. The National Job 
Corps Study, a nationally representative RCT, found impacts on earnings and hours worked in the fourth 
year after enrollment (Schochet et al. 2008). A follow-up to the National Job Corps study found some 
long-term beneficial effects for the older students (ages 20 to 24), with employment gains of 4 percentage 
points, a 40 percent reduction in disability benefit receipt, and a 10 percent increase in tax filing rates in 


® Some of these components are closely related to strategies discussed in previous sections, but this section focuses 
on the evidence of these strategies implemented specifically for youth populations. 
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2015 (Schochet 2020). However, these studies are 
based on youth entering the program in the mid- 
1990s; there are no recent, rigorous evaluations of 
the program. 


The YouthBuild program, also funded under 
WIOA, aims to provide at-risk youth ages 16 to 24 
with services to help them attain a high school 
diploma or equivalent and teaches them 
construction skills, and it has expanded to offer 
youth career pathways training in high-demand 
industry sectors (YouthBuild 2020). In a recent 
RCT of YouthBuild, approximately 75 percent of 
the youth randomly assigned to the program group 
(invited to enroll in YouthBuild) participated in 
the program, and of those who participated, 50 
percent graduated. The study also observed that 
YouthBuild improved employment relative to the 
control group (youth not invited to enroll in 
YouthBuild) during the second year after random 
assignment and led to a small increase in survey- 
reported wages and earnings at 30 months, but it 
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Box 5. Elements of WIOA Youth programs 


e Tutoring, study skills training, instruction, and 
dropout prevention 


e Alternative secondary school services or dropout 
recovery services 


e Paid and unpaid work experience 
e Occupational skills training 


e Education offered concurrently with workforce 
preparation and training for a specific occupation 


e Leadership development opportunities 

e Supportive services 

e Adult mentoring 

e Follow-up services 

e Comprehensive guidance and counseling 

e Financial literacy education 

e Entrepreneurial skills training 

e Services that provide labor market information 

e Postsecondary preparation and transition services 
Source: WIOA Sec. 121(b) 


had no impact on earnings from administrative data (Miller et al. 2018). 


Other evaluations of youth programs with comprehensive services that are not funded by WIOA have also 
produced some encouraging findings on youth’s employment and earnings outcomes. These programs, 
reviewed below, offered youth a comprehensive set of services, including education, occupational 


training, work experience, and other supports. 


e The National Guard Youth ChalleNGe program is a six-month residential program to improve the 
education, life skills, and employment potential of high school dropouts. Upon completing the 
residential phase of the program, participants receive a year of structured mentoring and are placed in 
employment, education programs, or the military. An RCT evaluation of the program found that three 
years after enrollment, program participants had higher rates of employment and earnings than the 


control group (Millenky et al. 2011).’ 


e Year Up provides urban young adults with intensive training and offers internship tracks in 
desktop/network support, quality assurance, project management, advanced system administration, 
and cyber security at corporate partner locations. An RCT of Year Up’s career pathways program for 
youth found that the program increased participant earnings, with program participants earning 
$1,895 more than those in the control group in the sixth and seventh quarters following random 


assignment (Fein and Hamadyk 2018). 


e Youth Transition Demonstration (YTD) projects, conducted between 2006 and 2012, were 
intended to help youth ages 14 to 25 with disabilities who received or were at risk of receiving Social 
Security disability benefits to become more economically self-sufficient. Participants were offered 


7 A more recent report on this program, Millenky et al. (2021), is an outcomes study and does not present evidence 


on impacts. 
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some combination of benefits counseling, career counseling, and coordination of services, as well as 
internships, job shadowing, job coaching, competitive paid employment, and empowerment training. 
An RCT study examining the program implemented in West Virginia observed positive impacts on 
employment and earnings one year after implementation (Fraker et al. 2012). However, an RCT 
measuring the impacts for all six YTD project sites 24 months after random assignment found no 
significant impacts on employment and earnings outcomes (Hemmeter 2014). 


e Youth Corps provides youth with a stipend as well as educational services, employment and training, 
and community service activities. An RCT study of Youth Corps found positive impacts on earnings 
18 months after enrollment but no impacts on employment or education (Price et al. 2011). 


e Youth Career Connect (YCC) is a high school-based program designed to encourage America's 
school districts, institutions of higher education, the workforce investment system, and their partners 
to scale up evidence-based high school models that will transform the high school experience for 
America's youth to connect them to college and careers. The program blends features of the career 
academy and sector-based models, including small learning communities; college preparatory 
curriculum based on a career theme that aligns occupational training with employer needs; and 
employer, higher education, and community partners (Maxwell et al. 2019). A study of YCC included 
two components, a QED in 16 school districts and an RCT in four school districts that were also part 
of the QED. Overall, the study found small positive impacts related to accumulating credits for high 
school graduation (Maxwell et al. 2019). 


Work-based learning, a strategy used in many programs funded by WIOA Youth to provide youth with 
occupational and basic skills training in a work-like setting, is found to have positive effects on 
employment and earnings outcomes (Carter et al. 2011). For example, an RCT study found that youth 
who participated in the Center for Employment Training (CET) were more likely to obtain a credential 
than those who did not participate in CET services. Additionally, participants were more likely to obtain 
employment in the sectors targeted by the program (Miller et al. 2005). 


Similarly, internship programs for youth and young adult populations, commonly used by programs 
supported by WIOA Youth, have also demonstrated some encouraging findings (Hollenbeck 1996; Quint 
et al. 1997); however, only one study was conducted within the last 20 years. The study focused on the 
Young Adult Internship Program and found that the program demonstrated increased earnings for 
participants a year after completing their internship (Skemer et al. 2017). 


A. Programs for specific populations of youth or with specific barriers to employment 


The WIOA Youth program has shifted from serving in-school youth to a greater focus on serving older, 
out-of-school youth, though many states have obtained waivers to use only 50 percent of their funds on 
out-of-school youth rather than the 75 percent expenditure specified in the legislation (Dunham 
forthcoming). In addition, changes to out-of-school youth eligibility have encouraged the program to 
serve youth who fall into one or more barrier-related categories, such as disconnection from school, 
involvement with the justice system, involvement in foster care, experiencing homelessness, pregnant or 
parenting, or disabled (Dunham forthcoming). The following studies assess strategies for specific 
populations of youth. 


1. Youth with justice system involvement 


Two studies assessing the impact of employment-related programs on youth with justice involvement 
observed positive findings. In a random assignment study of the Avon Park Youth Academy and 
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STREET Smart program (National Council on Crime and Delinquency 2009), the program improved 
employment and earnings. The Avon Park Youth Academy and STREET Smart program serves youth 
ages 16 to 18 who are transitioning out of a secure custody residential facility in Florida, and its 
vocational training component includes opportunities for work-based learning through on-the-job 
training. In a subgroup analysis of JOBSTART, a program providing basic skills education, occupational 
training, support services, and job placement assistance to youth who had dropped out of school, young 
males with prior justice system involvement experienced statistically significant gains in earnings in the 
fourth year after random assignment (Cave et al. 1993). A large-scale evaluation of the Reentry 
Opportunities Employment grant program is currently under way and will examine impacts on youth with 
involvement in the justice system. 


2. Youth with prior or current foster care involvement 


Although few studies have examined the labor market outcomes of youth with prior or current foster care 
involvement, two have found positive impacts. In an RCT of an intervention helping youth transition out 
of foster care, youth in the program earned an average of $611 more than youth who did not participate in 
the program (Valentine et al. 2015). Youth who participated in the program received counseling, referrals 
to other services, financial assistance, group social and learning activities, and educational and vocational 
coordination. In a QED study of the Foster Youth Demonstration Project, which provided youth with job 
preparation and educational and supportive services, youth with foster care involvement who participated 
in services longer were more likely than those participating for fewer quarters to secure a paid job 
(Institute for Educational Leadership 2008). 


3. Youth experiencing homelessness 


A study including two experiments found positive employment impacts of a social enterprise intervention 
and an Individual Placement and Support (IPS) program for homeless youth. An RCT comparing a social 
enterprise to the IPS program for homeless youth reported that 39 percent of youth who participated in a 
social enterprise program reported any paid employment, compared to 32 percent of youth in the IPS 
program over the 20-month study period. However, this difference was not statistically significant. 
(Ferguson 2018). The youth who participated in the social enterprise intervention attended vocational and 
small-business classes and received clinical or case-management services. Youth in the IPS program met 
individually with their employment specialist, a case manager, and a clinician at least once weekly to 
discuss life goals such as employment (Ferguson 2018). In another study of youth receiving shelter 
services including temporary housing, skills training, and referral services, youth who received these 
services showed no significant improvement in employment status when compared to those receiving day 
treatment (Thompson et al. 2002). 


4. Youth parents or expectant parents 


An RCT of the Young Parents Demonstration (YPD) examined the provision of enhanced services to 
improve educational and employment outcomes for youth parents and those expecting a child. The core 
program components offered to all study participants typically included education, training, and 
employment-focused services as well as supportive services. The enhancements offered to the treatment 
group included mentoring or guided employment, education, training, and related supports. The study 
found positive earnings impacts for the first two rounds of grantees through the first two years after 
random assignment. However, overall, the enhanced services had no impact on employment and earnings 
(Trutko et al. 2018). 
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5. Youth disconnected from education or employment 


The Performance Partnership Pilots provides services for disconnected youth, defined as individuals 
between ages 14 and 24 who are low income and either homeless, in foster care, involved in the juvenile 
justice system, unemployed, or not enrolled in or at risk of dropping out of an educational institution. In a 
synthesis of the local evaluations of the first cohort of pilots, of the six types of interventions 
implemented, three demonstrated evidence of improving expected youth outcomes—case management 
services for out-of-school youth, combined case management and WIOA services for out-of-school youth, 
and a two-generation education and training program for pregnant and parenting youth. However, one of 
the three local evaluations examining case management for disconnected youth also found evidence of 
negative effects of participation in career preparation and subsidized employment (Maxwell and Yafiez 
2020). 


6. Youth who have experienced trauma 


Two models of trauma-informed approaches for youth have been rigorously evaluated: the Attachment, 
Self-regulation, and Competency (ARC) model and the Sanctuary Model. The ARC model focuses on 
improving three areas impacted when youth experience trauma—attachment, self-regulation, and 
resiliency (Berk et al. 2018). For youth ages 13 to 19 in a residential environment, use of the ARC model 
reduced post-traumatic stress disorder and improved behavior (Hodgdon et al. 2013). The Sanctuary 
Model—initially developed for adults but more recently adapted to in-school youth and children— 
develops an understanding of trauma, uses a framework for addressing disruption, and includes an 
implementation toolkit (Berk et al. 2018). In a residential setting, one combined experimental and quasi- 
experimental study found that youth in locations that had implemented the Sanctuary Model had greater 
self-control, reduced verbal aggression, and used fewer negative coping strategies after six months 
(Rivard et al. 2005). In addition, trauma-informed approaches are also used for adults with barriers to 
employment in settings outside of WIOA, and this strategy could be applicable to some adult job seekers 
within WIOA programs. 


B. Knowledge gaps related to youth programs 


The studies of training models provided to youth demonstrated some positive impacts, particularly those 
that included some combination of work experience or supportive services in addition to training. The 
evidence on interventions serving specific populations of youth is limited and finds mixed results, and it 
does not always examine interventions provided exclusively to these populations. Also, in recent years, 
youth programs have shifted to serving youth who are out of school or have barriers to employment. The 
following gaps could be filled by future studies: 


What combination of training, case management, supportive services, follow-up services, and other youth 
program elements contribute to positive impacts for youth? How do models that provide comprehensive, 
sector-based education and training compare to models that provide more limited work experience? 


What are the impacts of programs directed specifically toward those out of school and those with barriers 
to employment? 


How do impacts on youth served by programs supported by WIOA Youth compare to impacts on similar 
youth participating in programs serving broader populations? 
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